Inertia-Constrained Pixel-by-Pixel Nonnegative Matrix Factorisation: a Hyperspectral Unmixing Method Dealing with Intra-class Variability

Blind source separation is a common processing tool to analyse the constitution of pixels of hyperspectral images. Such methods usually suppose that pure pixel spectra (endmembers) are the same in all the image for each class of materials. In the framework of remote sensing, such an assumption is no more valid in the presence of intra-class variabilities due to illumination conditions, weathering, slight variations of the pure materials, etc... In this paper, we first describe the results of investigations highlighting intra-class variability measured in real images. Considering these results, a new formulation of the linear mixing model is presented leading to two new methods. Unconstrained Pixel-by-pixel NMF (UP-NMF) is a new blind source separation method based on the assumption of a linear mixing model, which can deal with intra-class variability. To overcome UP-NMF limitations an extended method is proposed, named Inertia-constrained Pixel-by-pixel NMF (IP-NMF). For each sensed spectrum, these extended versions of NMF extract a corresponding set of source spectra. A constraint is set to limit the spreading of each source's estimates in IP-NMF. The methods are tested on a semi-synthetic data set built with spectra extracted from a real hyperspectral image and then numerically mixed. We thus demonstrate the interest of our methods for realistic source variabilities. Finally, IP-NMF is tested on a real data set and it is shown to yield better performance than state of the art methods

Toulouse Hyperspectral Data Set: a benchmark data set to assess semi-supervised spectral representation learning and pixel-wise classification techniques

Airborne hyperspectral images can be used to map the land cover in large urban areas, thanks to their very high spatial and spectral resolutions on a wide spectral domain. While the spectral dimension of hyperspectral images is highly informative of the chemical composition of the land surface, the use of state-of-the-art machine learning algorithms to map the land cover has been dramatically limited by the availability of training data. To cope with the scarcity of annotations, semi-supervised and self-supervised techniques have lately raised a lot of interest in the community. Yet, the publicly available hyperspectral data sets commonly used to benchmark machine learning models are not totally suited to evaluate their generalization performances due to one or several of the following properties: a limited geographical coverage (which does not reflect the spectral diversity in metropolitan areas), a small number of land cover classes and a lack of appropriate standard train / test splits for semi-supervised and self-supervised learning. Therefore, we release in this paper the Toulouse Hyperspectral Data Set that stands out from other data sets in the above-mentioned respects in order to meet key issues in spectral representation learning and classification over large-scale hyperspectral images with very few labeled pixels. Besides, we discuss and experiment the self-supervised task of Masked Autoencoders and establish a baseline for pixel-wise classification based on a conventional autoencoder combined with a Random Forest classifier achieving 82% overall accuracy and 74% F1 score. The Toulouse Hyperspectral Data Set and our code are publicly available at https://www.toulouse-hyperspectral-data-set.com and https://www.github.com/Romain3Ch216/tlse-experiments, respectively.Comment: 17 pages, 13 figure

p3^3VAE: a physics-integrated generative model. Application to the semantic segmentation of optical remote sensing images

The combination of machine learning models with physical models is a recent research path to learn robust data representations. In this paper, we introduce p$^3$VAE, a generative model that integrates a perfect physical model which partially explains the true underlying factors of variation in the data. To fully leverage our hybrid design, we propose a semi-supervised optimization procedure and an inference scheme that comes along meaningful uncertainty estimates. We apply p$^3$VAE to the semantic segmentation of high-resolution hyperspectral remote sensing images. Our experiments on a simulated data set demonstrated the benefits of our hybrid model against conventional machine learning models in terms of extrapolation capabilities and interpretability. In particular, we show that p$^3$VAE naturally has high disentanglement capabilities. Our code and data have been made publicly available at https://github.com/Romain3Ch216/p3VAE.Comment: 21 pages, 11 figures, submitted to the International Journal of Computer Visio

Hydrocarbon Pollution Detection and Mapping Based on the Combination of Various Hyperspectral Imaging Processing Tools

Oil extraction and transportation may lead to small or large scale accidental spills, whether at sea or on land. Detecting these spills is a major problem that can be addressed by means of hyperspectral images and specific processing methods. In this work, several cases of onshore oil spills are studied. First, a controlled experiment was carried out: four boxes containing soil or sand mixed with crude oil or gasoil were deployed on the ONERA site near Fauga, France, and were overflown by HySpex hyperspectral cameras. Owing to this controlled experiment, different detection strategies were developed and tested, with a particular focus on the most automated methods requiring the least supervision. The methods developed were then applied to two very different cases: mapping of the shoreline contaminated due to the explosion of the Deepwater Horizon (DWH) platform based on AVIRIS images (AVIRIS: Airborne Visible/InfraRed Imaging Spectrometer), and detection of a tar pit on a former oil exploration site. The detection strategy depends on the type of oil, light or heavy, recently or formerly spilled, and on the substrate. In the first case (controlled experiment), the proposed methods included spectral index calculations, anomaly detection and spectral unmixing. In the case of DWH, spectral indices were computed and the unmixing method was tested. Finally, to detect the tar pit, a strategy based on anomaly detection and spectral indices was applied. In all the cases studied, the proposed methods were successful in detecting and mapping the oil pollution

Classification of Hyperspectral Reflectance Images With Physical and Statistical Criteria

A classification method of hyperspectral reflectance images named CHRIPS (Classification of Hyperspectral Reflectance Images with Physical and Statistical criteria) is presented. This method aims at classifying each pixel from a given set of thirteen classes: unidentified dark surface, water, plastic matter, carbonate, clay, vegetation (dark green, dense green, sparse green, stressed), house roof/tile, asphalt, vehicle/paint/metal surface and non-carbonated gravel. Each class is characterized by physical criteria (detection of specific absorptions or shape features) or statistical criteria (use of dedicated spectral indices) over spectral reflectance. CHRIPS input is a hyperspectral reflectance image covering the spectral range [400–2500 nm]. The presented method has four advantages, namely: (i) is robust in transfer, class identification is based on criteria that are not very sensitive to sensor type; (ii) does not require training, criteria are pre-defined; (iii) includes a reject class, this class reduces misclassifications; (iv) high precision and recall, F 1 score is generally above 0.9 in our test. As the number of classes is limited, CHRIPS could be used in combination with other classification algorithms able to process the reject class in order to decrease the number of unclassified pixels

Onshore Hydrocarbon Remote Sensing

International audienceHydrocarbon detection is important for both environment monitoring and hydrocarbon exploration. Hyperspectral imaging and derived spectral indices are used to detect hydrocarbons. With appropriate indices, light hydrocarbons on bare ground are detected. Heavier hydrocarbons are more difficult to detect. Plastic items are very well detected. Shadows and vegetation are generating some false alarms. Detection of hydrocarbon in urban environment, or on bare soils will be possible using spectral indices while detection of hydrocarbon in remote vegetated country areas will be difficult

Performance comparison of geometric and statistical methods for endmembers extraction in hyperspectral imagery

International audienc

Cartographie automatique de littoral pollue par les hydrocarbures par imagerie hyperspectrale

International audienceLarge-scale mapping of coastal oil spills and their monitoring over time is a major issue that can be adressed by using hyperspectral images and dedicated processing. Previous researches have shown that it is possible to map the polluted coastline caused by the explosion of the Deepwater Horizon (DwH) platform from AVIRIS images (AVIRIS: Airborne Visible/InfraRed Imaging Spectrometer). But the detection processes required either ground truth or laboratory spectra of hydrocarbons or were not fully automatic. In this paper we focused on an AVIRIS image which covers The Bay Jimmy, located south of New Orleans, and particularly impacted by oil pollution. Two automatic methods were developed to detect oiled coasts. In the first one, we have developed a new spectral index able to detect directly hydrocarbon and less sensitive to noise than indices proposed in previous works. The second one extracts endmembers via Orthogonal Subspace Projection, and then sorts the endmembers in terms of hydrocarbon indices scores, in descending order. Then, the detection map or the abundance map corresponding to the best endmember is used to map oiled areas. Both approaches give results consistent with those of studies previously conducted on the same image, and with maps built from field observations.La cartographie à grande échelle des marées noires côtières et leur suivi dans le temps est un problème majeur qui peut être résolu en utilisant des images hyperspectrales et un traitement dédié. Des recherches antérieures ont montré qu'il était possible de cartographier le littoral pollué par l'explosion de la plate-forme Deepwater Horizon (DwH) à partir d'images AVIRIS (AVIRIS : Airborne Visible/InfraRed Imaging Spectrometer). Mais les processus de détection nécessitaient soit l’utilisation de mesures in-situ, soit des spectres d'hydrocarbures en laboratoire, ou bien n'étaient pas entièrement automatiques.Dans cet article, nous nous sommes concentrés sur une image hyperspectrale AVIRIS qui couvre La Baie Jimmy, située au sud de la Nouvelle-Orléans, et particulièrement affectée par la pollution. Deux méthodes automatiques ont été mises au point pour détecter les côtes mazoutées. Dans la première, nous avons développé un nouvel indice spectral capable de détecter directement les hydrocarbures et moins sensible au bruit que les indices proposés dans les travaux précédents. La seconde utilise une méthode de démélange spectral, puis trie les pôles de mélange en termes d'indices d'hydrocarbure par ordre décroissant. Les cartes de détection ou d'abondance correspondant au meilleur pôle de mélange sont alors utilisées pour cartographier les zones mazoutées. Les deux approches donnent des résultats cohérents avec ceux d'études réalisées précédemment sur la même image et avec des cartes établies à partir d'observations sur le terrain

Correction atmosphérique d'images hyperspectrales infrarouges et découplage émissivité-température de surface

Nous étudions l'estimation du spectre d'émissivité et de la température de surfaces au sol à partir d'images hyperspectrales infrarouges de résolution spatiale métrique. Nous considérons les luminances acquises à 8 km d'altitude en visée au nadir entre 4 et 12 m par un radiomètre dont la résolution spectrale est de 10 cm-1 au-dessus de 8 m et 15 en dessous. Notre approche utilise une nouvelle méthode de sondage atmosphérique de la bande d'absorption du CO2 à 4,3 m et de celle de la vapeur d'eau entre 5 et 8 m couplée à un algorithme de séparation émissivité/température. Le sondage atmosphérique est réalisé par deux jeux de réseaux de neurones paramétrés pour estimer les trois premiers coefficients de l'analyse en composante principale des profils atmosphériques de température et le contenu total en vapeur d'eau. Le choix de ces variables de sortie s'est appuyé sur une analyse de sensibilité conduite par un plan d'expériences. Le résultat du sondage atmosphérique permet ensuite de calculer les paramètres radiatifs entre 8 et 12 m nécessaires pour estimer les spectres d'émissivité et les températures des surfaces au sol. Nous étendons l'algorithme SpSm afin d'améliorer les résultats obtenus avec des profils en vapeur d'eau de forme atypique. Cette nouvelle méthode baptisée SpSm2D introduit en tant qu'inconnue la forme des profils de vapeur d'eau lors de l'estimation des températures de surface et spectres d'émissivité. La méthode est développée puis évaluée à partir d'un grand nombre de simulations effectuées pour les différents types de masse d'air de la base de profils atmosphériques TIGR2000, différents angles solaires, les spectres d'émissivité de la base ASTER et des températures des surfaces au sol choisies aléatoirement. On étudie également l'influence de différents bruits de mesure et l'on présente une méthode adaptée au traitement d'images infrarouges exploitant l'homogénéité spatiale de l'atmosphère. Les résultats montrent que l'on parvient à estimer les températures de surface et les spectres d'émissivité à 1.5 K et 3% près respectivement. La méthode en général et l'algorithme de sondage en particulier se révèlent également robustes aux différents bruits testés. En outre, la prise en compte de l'homogénéité spatiale de l'atmosphère améliore sensiblement les résultats du sondage et donc du processus d'estimation en général. La méthode est finalement testée sur les données du capteur S-HIS acquises lors de la campagne de mesure EAQUATE.TOULOUSE-ISAE (315552318) / SudocSudocFranceF